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^ ; Abstract 

\^ • The paper presents a lightweight discussion of relations between energy, entropy, information, 
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and intelligence, based on an analysis of the energy needed for computation. 
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I. INTRODUCTION 



This paper is a remake and an extrapolation of ideas presented in the chapter "Reversible 
computation and the thermodynamics of computing" in a beautiful Feynman book.- Basi- 
cally, it is about energetics of computing. A computer is a physical device, whose dynamics 
we use for computing. Therefore the question on energy requirements is certainly relevant. 
This question has been with us since the time of von Neumann, when computers were just 
born. The discussion on minimal energy required for computing culminated in 1960 's and 
70's in the works of Landauer-, Bennett^, Toffoli^ and others. The review by Bennett^ can 
be consulted for details. Finally, the discussion converged on the recognition that reversible 
dynamics can be used for computing purposes and on the fact that when used, energy needs 
to be dissipated only during the process of memory resetting (sometimes the term "erasure 
of information" is used, even though it may be confusing). We shall repeat the Feynman 
argument why energy is needed to reset memory into the default state in Section [Tll We 
shall also dwell on discussion of a complementary process: how information can be used to 
extract useful work. The line of reasoning will proceed along the following points. 

• Resetting memory requires energy. 

• Energy needed for memory resetting is proportional to the amount of erased informa- 
tion. 

• A device able to perform alternative actions needs memory resetting to be reusable. 

• Information (knowledge) can be used to extract useful work. 

• A device able to extract "work from information" needs to spend the same amount of 
work to be reused. 

• Intelhgent inductive inference (plus a good deal of luck) enables to extract useful work 
without the need for resetting. 

II. RESETTING MEMORY REQUIRES ENERGY 

Feynman invented a simple physical model of a one-bit register, presented in Fig. [1] It 
is a cylinder divided into two compartments by a central partition. The cylinder contains 
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just one molecule, which can be either in the left compartment (representing logical 0) or 
in the right compartment (representing logical 1). Let us define the default register state 
to be "0" (the molecule in the left compartment). It is not without interest that when we 




FIG. 1. Feynman Register. 



know the actual state of the register, we can bring it to the default state "0" (reset it) 
without spending energy. The process of resetting a register from a known state (we shall 
call it qualified resetting) is graphically presented in Fig. |2l If the known actual state is 
"0", we need not do anything. If the known state is "1", we use the central partition as 
a piston, joining it firmly with the piston attached to the cylinder from the right side and 
move the two pistons together to the left end. This operation does not require performing 
mechanical work from our side, since the forces of the (one-molecule) gas acting on the two 
pistons are opposite and equal in size. Finally we convert the piston in the central position 
into a partition and close the cylinder at the right end. Voila, the register is in the default 
state and it has cost no energy. This is not the end of the story, we shall add an important 
discussion in Section HVl 



FIG. 2. Resetting to the default state from a known state (qualified resetting). 



Now let us discuss how to bring the register to the default state if we do not know 
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its actual state (unqualified resetting). The method is presented in Fig. O We start by 





FIG. 3. Resetting to the default state from an unknown state (unqualified resetting). 



pulling the central partition aside. Then pushing a piston from the right side until the 
central position brings the molecule into the left compartment. For this, however, we need 
to perform mechanical work, since the (one-molecule) gas acts on the piston by its pressure 
p. The calculation of the needed amount of work W is easy. 

rV/2 pV/2 ,rjn 

W = - pdV = - — = -kT{\n{V/2) - \n{V)) = kT\n2. (1) 
Jv Jv ^ 

Some notes: If we were calculating the work performed by the gas, there would be no minus 
sign. We are, however, calculating the work performed by an external hobbit pushing the 
piston, acting by force opposite in direction to that of the gas force and therefore the minus 
sign. We have used the gas state equation for = 1 molecules: pV = NkT. 

What happens with the work A;Tln2? We have effectively assumed that the cylinder is 
in contact with the environment having temperature T. In order not to increase the gas 
temperature, the energy kT ln2 has to be dissipated into the environment as heat. 

Summarizing: we have shown that erasing (destroying) 1 bit of information costs energy 
/cT In 2 to be dissipated into environment. We did not discuss in detail any other operation 
with memory registers like setting the register into desired memory state starting from the 
default state or copying the state of some register to another register, but it is known that 
those operations do not require performing work.- 

4 



III. INFORMATION, ENTROPY, ENERGY 



In the previous section, we have manipulated gas of a single molecule. The results 
extrapolated to gas of N molecules bring a new insight into the problem. We have seen 
that to compress gas, we have to perform work. On the other hand, if gas is expanding 
isothermally, it can perform useful mechanical work. Similar calculation as in Eq. [1] gives 
the amount of work performed by gas doubling its volume as 

W' = NkT ln2. (2) 

Somebody has to pay for this work: the environment in the form of heat 

Q' = NkT \n2. (3) 

However, during the gas expansion something else happens behind the scenes: our knowl- 
edge about the gas microstate decreases. Let us discuss it in more detail. Gas state is macro- 
scopically given by a few parameters, such as pressure p, volume V and temperature T. We 
say we have specified the gas macrostate. Microscopically the gas state is a tremendously 
complex notion. Describing the microscopic state requires giving the position and velocity 
of each individual molecule (we neglect quantum-mechanical complications). So knowing 
the macrostate, there is a good deal of information we are missing about the actual gas 
microstate. Now we shall argue that by expansion the amount of this missing information 
increases. 

Let us consider for simplicity a two-molecule gas that is doubling (isothermally and 
reversibly) its volume as presented in Fig. |H At the left side of the figure the original 




FIG. 4. The increase of the number of microstates by expansion. 

and final states are presented as macrostates. At the right side those states are presented 
as microstates. For simplicity we neglect molecular velocities, so a microstate is given by 
positions of all the molecules. In order to record positions of molecules, the gas volume is 
divided into smaller (virtual) compartments, six are presented (as an oversimplified example) 
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for the original state, 12 for the final state. In the original state placing two molecules into 
6 compartments is possible by (6 x 5)/2 = 15 ways. In the final state the number of possible 
microstates is (12 x ll)/2 = 66. So to communicate the exact microstate for the macrostate 
with smaller volume one has to specify a number between 1 and 15, to communicate the 
microstate for the doubled volume one has to specify a number between 1 and 66. Since 
there are more possibilities for the large volume, it is natural to say that if we do not know 
the exact microstate, the missing information is larger for larger volume. 

It is not difficult to make the statement more quantitative. We can play the same game 
with virtual compartments, just their number C must be very very large. For molecules 
in C compartments the number n of possible microstates is 

C! 

" {C-N)\NV 

If I want to communicate to you which microstate actually realizes the macrostate you 
observe, I have to send you its id number i: a number between 1 and n. The message would 
have ~ log2i bits, since that is the number of binary digits representing the number i. The 
mean number of bits sent will be 



I = log2 ^ = - loga i = - log2 n\ = - {log^ e) In n! (5) 

i=l 

For 



C > AT > 1 (6) 

we can use (several times) Stirling approximation In n! ^ n\nn — n and combining Eqs. (jl]) 
and dS]) we get 

/^iV(log2e)ln^ (7) 

If we decide to measure the information content of a message by its number of bits (we 
can do that if the coding used was optimal), we can conclude that if we know just the 
gas macrostate and not the microstate, we are missing information of I bits on average. 
Physicists prefer to measure information not in bits but in nats just omitting the factor 
logaC. Somewhat surprisingly, however, physicists found, that they already had defined a 
phenomenological physical quantity, the entropy S*, which differs from / expressed in nats 
only by another information- unit-changing factor k (the Boltzmann constant). So finally we 
get for the gas entropy 

C 

S ~ kN In — + term which takes into account molecular velocities (8) 



To repeat: S measures the amount of missing information (in units nats x Joule/Kelvin) if 
we know only the gas macrostate and not its microstate. 

So let us come back to the problem of expanding gas. Doubling the volume means 
doubling the number of compartments and therefore an increase of entropy 

2C C 
AS = kNln kNln— = kNln2 

N N 

The terms corresponding to molecular velocities canceled out, since the process is isothermal, 
and statistically the molecular velocities do not change. Increasing the volume lead to two 
effects 

• extracting heat from the environment and changing it into useful work, and 

• increasing the amount of missing information or, equivalently, decreasing the amount 
of our information about the gas microstate 

So we see that decreasing the amount of our knowledge enables us to extract heat ([3]) 
from the environment and change it into useful work ([2]). We shall discuss it in details in 
later sections. 

Comparing equations ([9]) and ([3]) we find the relation 

Q = TAS (10) 

Equation (fTOl) is generally valid for any reversible process in nature, we have observed just 
one very special case. 

Now we understand why we have to pay by performing work and dissipating heat in the 
process of unqualified resetting of a memory register into the default state (see Equation 
([T])). Before the proces of memory reset we did not know the gas state (the position of 
the molecule), after the proces we know that the register is in the default state (logical 0, 
molecule in the left compartment). So the amount of our knowledge increased. Therefore 
the entropy decreased, AS < so Q = TAS < 0. So the acquired heat is negative: heat 
has been dissipated. 
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(9) 



IV. ABILITY TO PERFORM (REPEATEDLY) ALTERNATIVE TASKS RE- 
QUIRES ENERGY DISSIPATION 



We have shown that quahfied resetting of memory requires no work, while unquahfied 
resetting of one bit of memory requires dissipation of heat Q = kT ln2. A question might 
arise: why do we usually perform unqualified memory resetting "as if" from an unknown 
state? In many cases, the register to be reset contains the result of the calculation. Most 
probably we have printed its content on a piece of paper, so we know its state and we should 
be able to do qualified resetting. 

The reason is that the device able to perform only unqualified memory resetting is simpler 
than the device able to perform qualified memory resetting. Look again at the Figures 
[2] and [31 The unqualified resetting can be performed by a "single-purpose hobbit" . The 
hobbit is trained to come from the right side to the register and push the piston. The 
qualified resetting needs a hobbit able to perform two alternative tasks. One is "do not do 
anything" , the other is "bind the pistons and push them from the right" . 

We shall argue that, quite generally, the "qualified hobbit" needs to have internal mem- 
ory register telling him which of the two alternatives to perform. In Fig. [5] we present a 
mechanical device (programable "processor") harnessing two "unqualified hobbits" to per- 
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FIG. 5. A general mechanical device enabling to harness two unqualified hobbits to perform 
qualified work. In the upper part the device is in the idle state, all the pistons are blocked, the 
hobbits are confined in their cages. In the lower part the device was put to action, the pistons were 
unblocked. Now one of the two cages is opened depending on the state of the internal Feynman 
memory register. 



form alternative works. There are two cages (left and right) with two unqualified hobbits in 
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them. The doors of the cages can be manipulated by the pistons, which feel the pressure of 
a Feynman one-molecule memory register. 

Let us describe how the device is operated by a supervisor. If the device is in the idle 
state, than all the pistons are blocked, and the hobbits are confined in their cages. Now the 
supervisor decides to use the device to perform work for which the left hobbit is trained. 
He has to set the internal Feynman memory register of the device to the "0" state (this is 
how the supervisor programs the device) and to unblock all the pistons. If the register is 
in the state "0", then only the left piston feels the pressure. It moves just a little, opens 
slightly the door of the left cage. The left hobbit escapes through the small opening and 
performs its task (and is trained to come back into the cage as a part of his "single-purpose 
algorithm" ) . The "working place" of the hobbit is not depicted in the Fig. |5l For example 
if the device is used to perform a qualified reset of some register, the hobbit leaves his cage, 
walks to the register (not depicted in the figure), performs the work he is trained for and 
returns back to his cage. Were the internal register in state "1" , it would be the right hobbit 
to perform his task. We have shown that a one-bit internal memory register can be used (by 
the supervisor) to set which of the two alternatives to perform. Since the doors are opened 
only slightly, only infinitesimal work is performed by the gas and this can be neglected. 
Therefore we do not need additional work to perform alternative tasks. 

But wait a moment! After the work was done, both hobbits are back in their cages, the 
pistons are blocked, the device is in the idle state. However, the internal register remains 
"armed for the left". If the supervisor does nothing with that, the machine effectively 
becomes a "single purpose" (unqualified) device performing in each work cycle just the "left 
algorithm" . If the supervisor wants to use the device as a qualified device in the next work 
cycle, he has to reset its internal register. After resetting, the register is prepared to be 
programmed by copying new information into it. Unqualified resetting costs work, copying 
new information needs no additional energy costs. We stress that the register reset should 
be done in an unqualified way, otherwise additional qualified-work-able device should be 
used for the resetting. This additional device has to have its internal register which itself 
needs to be reset after work and the chain would grow indefinitely unless at some stage an 
unqualified reset is used. 

Here is the point where a thorough reader might stop and suggest: why the hobbits do not 
reset their supervising register themselves (as the last part of their work). The left hobbit 
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will do nothing (the register being in the default state), the right hobbit will do the two- 
joined-pistons-push operation (see Figl2]). Qualified resetting done, no energy dissipation 
needed! Unfortunately, this cannot be done. The discussion is presented in the Appendix. 

Now we take courage and generalize the result. We conjecture that any device able 
to perform repeatedly qualified (two alternatives) work has to contain an internal one-bit 
memory register, which should be reset in an unqualified way. This needs heat Q = kT In 2 
to be dissipated into the environment for each working cycle. We cannot use qualified 
resetting: for that we would need another qualified device which itself should have internal 
register to be reset, so we would just push the energy expenditure one step further. 

Conclusion: memory reset needs to be done in an unqualified way and costs energy kT In 2 
per bit. It is worth to stress that once the internal register of the device is in the default 
state, no work is needed to set it into desired state or copy to it the information content of 
some other register. So "programming the qualified hobbit" costs zero energy. 



V. IS IT POSSIBLE TO "EXTRACT WORK FROM INFORMATION"? 

In this section we shall discuss the opposite process to memory resetting: "extracting work 
from information". The original idea is due to Bennet^ ("a tape full of zeros can act as a 
'fuel' doing useful thermodynamic or mechanical work as it randomizes itself"). 

The principle is explained in Figs. [6] and [7l Suppose I have a one-bit register and I know 




FIG. 6. Preparations to extract work. 
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FIG. 7. Process of work extraction leading to loss of information. 

its state. I call a hobbit and send him to the register from the right side if the register is in 
state "0" and from the left side if it is in state "1". The hobbit mounts his handle to the 
central partition making a piston of it. Then I unblock the partition and the gas starts to 
expand performing work. The hobbit is consuming that work using it for any useful purpose. 
When the partition-piston arrives at the end of the cylinder and I insert a new partition to 
the central position (inserting partition does not cost any work!), I do not know at all in 
which compartment (left or right) the molecule is caught. The initial information was used 
to extract useful work, while the register was randomized. So the entropy of the register 
increased by 

A^ = fcln2 (11) 



the hobbit extracted the work 

W = kT\n2 
while the environment payed for that by heat 

Q = kT\n2 



(12) 



(13) 



used to keep the temperature constant. So the information helped me to exchange a chaotic 
form of energy (through heat) for useful ("organized") mechanical work. So if somebody 
sends me the whole train of N registers (see Fig. [8]) together with a delivery note stating for 
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FIG. 8. Sending information as a demonstration of the famous Landauer's quotation "information 
is physical" . 



each register its memory state, I can extract from it the work 

W = NkT ln2. (14) 

Good news! 

But wait a moment! To extract work from the train I need an army of quaUfied hobbits! 
Quahfied, since they must be able to perform alternative procedures (come to the register 
either from the left side or the right side). We have learned already that for that they need 
to have memory registers "in their heads", instructing them either to go to the left or the 
right. If I want to use the same hobbits to unload the next train, I have to reset their 
registers, and for that I have to pay by spending the work 

W = NkT\n2 (15) 

This is exactly the same work which I got by unloading the previous train. The net result 
is just zero. Bad news! 

There seems to be a way out. My energy supplier has to send trains (Fig. |9]) containing 
N registers each in the same memory state, for example "0". (This is what Bennett was 
saying in the quotation at the beginning of this section.) To extract energy, unqualified 
(single-purpose) hobbits are sufficient. Such hobbits do not have a register to be reset after 
work, so net work gain is positive. Good news! 
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FIG. 9. Uniform registers providing extractable energy source. 



But wait a moment! Now it is the energy supplier who might have energetic problems. 
He has to load the train "with zeros exclusively" . If he has access only to a pile of registers 
in random states, he has to check (measure) the state of each register and then to load it or 
not to load it according to the findings. This means alternative processes: to load or not to 
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load (yes, Hamlet also needs a register!), so now the energy supplier needs qualified hobbits 
with registers needing reset. So the supplier spends the same amount of work as it is gained 
at the side of a customer. So the total net amount is again zero. Bad news! 
But wait a moment! There is a chance described in the next section. 

VI. INTELLIGENCE AND INDUCTIVE INFERENCE 

The reader should be warned that this section contains really a lightweight discussion. 
It is more entertainment then science, although I believe deeper aspects are hidden behind 
the scenes. 

We start our discussion with a seemingly unrelated topics: tests of intelligence. (Well, 
that is a very controversial subject itself.) The first (and easiest) question one meets in the 
intelligence tests is the following 

2, 4, 6, 8, ... What is the next number in the sequence? 

If you answer 2, 4, 6, 8, 10 you pass the test. If you instead write 

2,4,6,8,34 (16) 

you fail. But why, actually? Maybe you have realized, that the formula 

- lOn^ + 35n^ - 48n + 24 (17) 

reproduces the first four numbers and the extrapolation gives 34! So maybe you are more 
intelligent than some other candidate with his rather simple-minded formula 

2n. (18) 

(Clever people failing in intelligence tests stress this as a controversial matter for sure.) The 
test tries to evaluate your ability of inductive inference, and prefers simple solutions to more 
complicated ones. The simple solutions are considered to be more intelligent. Maybe there 
is a good reason for that but instead of trying to understand formal theory of inductive 
inference^, let us for simplicity conjecture that this preference is of Darwinian origin. To 
survive in a jungle, you must be able to find quickly solutions to problems you meet. If your 
mother teaches you a simple rule "big mane, run away" it helps you to survive better than if 
she gives you a copy of the complete lion genome to identify the animal. Relying on simple 
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solutions, you need luck to survive. With complicated solutions you would not survive for 
sure. So maybe we are offsprings of beings relying on simple inductive inference and that is 
why we prefer simple solutions to more complicated ones. 

Similarly, in a business jungle, if you make some ten control drills and each time you 
find a coal layer, you would consider to start a cole mine business. You simply bet that the 
Creator is not joking by distributing there just ten buckets of cole. Well, a bet is an hazard, 
the inductive inference is also a sort of an hazard, but what else you can do? 

So let us come back to our energy supplier that wants to make money by loading trains 
with Feynman memory registers in state "0". What he can do is to prospect for piles of 
Feynman registers and take a few samples from each pile found. If all the samples from a 
particular pile would be "state 0" , he can infer "this is a pile of uniform Feynman registers 
in state 0". He hires an army of cheap unqualified hobbits (no need to reset their registers!) 
and starts to deliver uniform (hopefully) information trains. With a piece of luck the net 
total result would be positive extracted useful work. 

VII. CONCLUSION 

Bet on intelligent inductive inference. Hopefully you will survive. 

Appendix: Impossibility of self-resetting 

Here we discuss the idea suggested in Section [IV] that the two hobbits ("left" and "right") 
could perhaps reset their supervising register themselves. Well, here we pay the prize for the 
lightweightness of our discussion. We used hobbits to make the explanations comprehensible. 
By that, however, we abandoned scientific rigor. Behind the scenes everywhere in our 
discussion is a hidden assumption: hopefully one can substitute the lightweight argument 
by a rigor one. 

Hobbits escaping from cages performing some work should somehow correspond to strictly 
reversible-dynamics realization of the computing tasks. Reversibility is essential if we do not 
want to dissipate additional energy by the "working hobbits" . We did not even try to support 
this hope for reversible realization. If reversible implementation were not possible, it would 
just mean that we have to dissipate even more energy, what does not spoil our argument 
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that the abihty to perform (repeatedly) alternative tasks requires energy dissipation. 

We shall argue now that the self-resetting-hobbit scheme, though being so attractive, is 
not reversibly realizable. The essence of our argument can be found in Bennett's paper.- The 
flow chart of the self resetting scheme is presented in Fig. [TDl In the point X the hobbits are 
back in their cages and the register is reset. That means that in the point C we cannot tell, 
whether we have got there through the branch A or B. It means that the scheme is logically 
irreversible (we cannot "undo" the calculation). Logical irreversibility means dynamical 
irreversibility, what means energy dissipation. Self resetting is not possible without energy 
dissipation. 




FIG. 10. Flowchart of the self-resetting scheme 
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